Cameras generally capture light from a scene to produce an image of the scene. Some cameras can also capture depth or disparity information. These multi-mode cameras are becoming more ubiquitous in the environment, from mobile phones, to gaming systems, etc. Generally, the image data is provided separately from the depth data to consuming applications (e.g., devices, software, etc.).